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DETAILED ACTION 

Response to Amendment 

1. The amendments, filed on 06/18/2008, have been entered and made of record. Claims 1-4, 6- 
11, 13-15 and 17-32 are pending. 

Response to Arguments 

2. Applicant's arguments with respect to claims rejected under 35 U.S.C. § 103 have been 
considered but are moot in view of the interpretation of the original cited references. 

3. Applicant's arguments, see pages 10-14, filed on 06/27/2008, not clarify made with respect 
to which claim(s). For the purpose of argument, the Applicant remarks on pages 10-14 will 
be read as referring to independent claim 1. 

4. In response to applicant's argument that the references fail to show certain features of 
applicant's invention, it is noted that the features upon which applicant relies (i.e. 
"operations that occur during the capture process") are not recited in the rejected claim(s). 
The applicant is reminded that although the claims are interpreted in light of the 
specification, limitations from the specification are not read into the claims. See In re Van 
Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). The specification describes 
various computer-implemented operations involving data stored in computer systems. On 
page 1 1 of Applicant remark dated 06/27/2008, Applicant fails to point out which/what 
operations that occur during the capture process. Examiner has carefully reviewed the 
descriptive portion of the specification and still failed to locate any mention of the any 
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operation that occur during the capture process. The Applicant is kindly asked to explain 
where a particular "operation" is locate or suggest that explain the operation that occur 
during the capture process. Therefore, independent claim 1 is obvious over by Gvili in view 
of Parker. Applicant's arguments are not convincing. 

5. The applicant argues that neither Park's new teachings, nor the teachings of Gvili et al. teach 
or suggest the claimed depth mask for identifying objects within the foreground region with a 
first bit value and identifying objects within the background region with a second bit value. 
The examiner understands the applicant's arguments but respectfully disagrees with the 
applicant's assessment. In response to applicant's argument, it is noted that Examiner used 
Park's reference to disclose the function of adjusting the image capture device parameters 
and not for identifying objects. It is noted that Gvili discloses a real-time computation of this 
algorithm was implemented on a Pentium 3 machine (page 572, 5 th paragraph). Therefore 
Gvili reference discloses the depth mask identifying objects within the foreground region 
with a first bit value (full value pixels representing foreground pixels in the color channel) 
and identifying objects within the background region with a second bit value (zero value 
pixel representing background) (page 564, 6 th paragraph) during the image capture process. 
Accordingly, independent claim 1 is obvious over by Gvili in view of Parker. Applicant's 
arguments are not convincing. 

6. The applicant further argues that the teachings of Parker do not add to what is discussed in 
Gvili et al. The examiner understands the applicant's arguments but respectfully disagrees 
with the applicant's assessment. In response to applicant's arguments against the references 
individually, one cannot show nonobviousness by attacking references individually where the 
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rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 
USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 
1986). As the Parker reference suggested there may be a large number of features are 
extracted for each segmented region in the image to represent a wide variety of visual 
saliency properties, which are then input into a tunable, extensible probability network to 
generate a belief map containing a continuum of values, thus the combination of Parker and 
Gvili will generate an enhanced compressed digital image (5:14-24, Parker). 

7. With respect to the independent claims 9, 17 and 25, and the dependent claims, 2-4, 6-8, 10- 
11, 13-15, 18-24, and 26-32, the examiner believes his office action of 03/24/2008 is proper 
and accurate. Applicant's arguments are not convincing. 

Claim Rejections - 35 USC § 103 

8. The text of those sections of Title 35, U.S. Code not included in this action can be found in a 
prior Office action. 

9. Claims 1, 3-4, 6-11, 13-15 and 17-32 are under 35 U.S.C. § 103(a) as being unpatentable 
over Gvili, "Depth Keying" SPIE Vol. 5006 (2003) in view of Parker, US 7,106,366. 

Regarding claim 1 , Gvili discloses a method for differentiating between foreground 
objects and background objects within a scene being captured through an image capture 
device (a novel depth video camera), comprising: 

- emitting a ray of light from a light source toward an object of the scene 
(generating a light wall, Para 3.1, pp 566-567); 
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opening an aperture cover allowing access to a sensor of the image capture 
device for reflected light from the light source (deploying a fast image shutter 
in front of the CCD chip, page 567, lines 3-4); 
- closing the aperture cover after a set time (the shutter is precisely controlling 
the exposure time of the CCD, page 567, lines 17-18), the predefined amount 
of time corresponding to a maximum distance traveled by the light (a real- 
time trimap is generated for each frame based on the original depth matte, 
page 569, lines 22-23); 

generating a depth mask identifying objects within a foreground region and a 
background region of the scene (foreground objects can be generated by 
setting the depth measurement window, page 568, section 3.2) based upon the 
light captured during the set time (a real-time trimap is generated for each 
frame based on original depth matte, page 569, 4th paragraph), the depth mask 
identifying objects within the foreground region with a first bit value (full 
value pixels representing foreground pixels in the color channel) and 
identifying objects within the background region with a second bit value (zero 
value pixel representing background) (page 564, 6 th paragraph); and 
adjusting image capture device parameters according to bit values of the depth 
mask prior to capturing a subsequent corresponding image of the scene 
(normalized depth of pixel D(iJ) can be calculated, page 567, lines 7-1 1) 
wherein the image capture device parameters are selected from one of focus 
(zoomed, page 568 line 2), brightness (collected light by each pixel, page 567, 
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line 5), exposure (exposure time, page 567, line 18) or gain (normalized depth 
of pixel, page 567, line 8). 
Gvili does not specifically teach that the adjusting of the image capture device parameters 
being done independently in the foreground from the background. 

However Parker does teach the adjusting of the image capture device parameters (digital 
image data outputs from sensor signal processing operation 204 for calculation of belief map 
209, Fig 3) being done independently in the foreground from the background (main subject 
detection unit 208 has a number of subtasks including region segmentation, perceptual 
grouping, feature extraction, and probabilistic reasoning for the subjects in the foreground) 
(5:14-67 and Figs 2a, 2b, 2c, and 3). 

Thus, it would have been obvious to one of ordinary skill in the art to have included a 
main subject detection unit as taught by Parker into Gvili 's digital camera, as the Parker 
reference suggested there may be other information relating to the importance of the 
photographed subject and corresponding background regions that can be used by the main 
subject detection unit to refine the belief map (8:20-34, Parker). 

Regarding claim 3, Gvili discloses the light source is configured to emit infrared light 
(IR laser diodes, page 567, lines 15-16). 

Regarding claim 4, Gvili discloses a method operation of opening an aperture cover 
allowing access to a sensor of the image capture device (the shutter is precisely 
controlling the exposure time of the CCD, page 567, lines 17-18) includes, receiving 
reflected light from the objects within the foreground region (light reflected from every 
object inside the depth measurement window, page 568, second paragraph of section 3.2). 
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Regarding claim 6, Gvili discloses a method operation of adjusting image capture 
device parameters according to bit values of the depth mask prior to capturing a 
subsequent corresponding image of the scene (ability to change the parameters of the 
depth window according to the scenario's need, page 568, section 3.2, first paragraph) 
includes, determining an optimal amount of light based upon the depth mask; and 
adjusting the aperture cover to allow the optimal amount of light into the image capture 
device (normalized depth of pixel can be calculated, page 567, lines 7-1 1). 

Regarding claim 7, Gvili discloses the image capture device parameters are adjusted 
through mechanical adjustments (depth key setting can be automatically set, page 568, 
third paragraph of section 3.2). 

Regarding claim 8, this claim recites same limitations as claim 3. Thus it is analyzed 
and rejected as previously discussed with respect to claim 3 above. 

Regarding claim 9, Gvili discloses a method for adjusting image capture settings for a 
single image capture device (a novel depth video camera), comprising: 

- identifying a scene (easily identified, page 569, third paragraph); 
capturing an image of the scene through the single image capture device (the 
depth information is captured by a camera, page 566, fifth paragraph); 
generating a depth mask of the scene from data defining the image of the 
scene (generating depth map according to some depth criterion, page 568, last 
paragraph); and 

- adjusting pixel values of the data defining the image corresponding to objects 
within any one or both of a foreground region and a background region of the 
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captured image (normalized depth of pixel D{i,j) can be calculated, page 567, 
lines 7-11), and the adjusting of pixel values is according to bit values of the 
depth mask (full value pixels representing foreground pixels in the color 
channel and zero value pixel representing background) (page 564, 6 th 
paragraph). 

Gvili does not specifically teach that adjusting pixel values associated with the 
foreground region are independent of adjusting of pixel values associated with the 
background region. 

However Parker does teach adjusting pixel values associated with the foreground region 
are independent of adjusting of pixel values associated with the background region (main 
subject detection unit 208 can adjusting pixel values associated with the foreground region 
independently because 208 has a number of subtasks including region segmentation, 
perceptual grouping, feature extraction, and probabilistic reasoning for the subjects in the 
foreground only) (5:14-67 and Figs 2a, 2b, 2c, and 3). 

Thus, it would have been obvious to one of ordinary skill in the art to have included a 
main subject detection unit as taught by Parker into Gvili 's digital camera, as the Parker 
reference suggested there may be other information relating to the importance of the 
photographed subject and corresponding background regions that can be used by the main 
subject detection unit to refine the belief map (8:20-34, Parker). 

Regarding claim 10, Gvili discloses the method operation of generating a depth mask 

of the scene from data defining the image of the scene includes segmenting the 
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foreground and background regions of the scene (automatically segmentation of the scene 
is possible, page 568, third paragraph of section 3.2). 

Regarding claim 1 1 , Gvili discloses the data defining the image of the scene includes 
pixel data where each pixel is tagged with distance information (calculate color distances 
between neighboring pixels, page 570, section d). 

Regarding claim 13, Gvili discloses the image capture device is selected from the 
group consisting of a digital camera, a web cam, and a camcorder (a novel depth video 
camera, page 564, line 4). 

Regarding claim 14, Gvili discloses the displaying a portion of the image of the scene 
having adjusted pixel values (a new matte is generated by combing the information from 
both the depth and the color frames, page 569, fourth paragraph and also page 571, first 
paragraph of section 4). 

Regarding claim 15, Gvili discloses the portion of the image of the scene is an image 
of a participant for use in an interactive gaming application (man-machine interactions, 
page 572, second paragraph of section 5). 

Regarding claim 17, this claim differs from claim 9 only in that the claim 9 is a 
method claim whereas claim 17 is an apparatus. Thus the apparatus claim 17 is analyzed 
and rejected as previously discussed with respected to claim 9 above. 

Regarding claim 18, Gvili discloses the depth mask is a bit mask having a first logical 
value (full value pixels) assigned to represent the foreground objects and a second logical 
value (zero value pixels) assigned to represent the background objects (full value pixels 



Application/Control Number: 10/758,817 Page 10 

Art Unit: 2622 

representing foreground pixels in the color channel and zero value pixel representing 
background) (page 564, 6 th paragraph). 

Regarding claims 19, this claim recites same limitations as claim 4. Thus it is 
analyzed and rejected as previously discussed with respect to claim 4 above. 

Regarding claim 20, Gvili discloses each logic element is one or a combination of 
hardware (i.e. shooting lens, depth sensor, as well as a camera) and software (i.e. a depth 
map, depth measurement window, and scene segmentation) (pp. 568-569, section 3.2 
depth key setting) 

Regarding claim 21, Gvili discloses the image capture device is a video capture 
device (a novel depth video camera, page 564, line 4). 

Regarding claim 22, Gvili discloses the depth logic is further configured to 
periodically provide a depth mask for a sequence of video frames captured by the video 
capture device (step f of the process for iteration of alpha values, page 570). 

Regarding claim 23, Gvili discloses the image capture device setting is adjusted 
through one of a mechanical or electrical adjustment (depth key setting can be 
automatically set, page 568, third paragraph of section 3.2). 

Regarding claim 24, Gvili discloses the image capture logic is further configured to 
adjust each pixel of image data of the scene (step d of the process shown the color 
distance calculation is performed for each pixel of the image data of the scene)( step d of 
the process for iteration of alpha values, page 570). 

Regarding claim 25, this claim differs from claim 1 only in that the claim 9 is a 
method claim whereas claim 25 is an apparatus. Claim 25 further differs from claim 1 in 
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that the limitations "a computing device" and "a display device" are additionally recited. 
Gvili teaches a computing device (a Pentium 3 machine, page 572, section 5) and a 
display (a video camera, page 564, line 4). Thus the apparatus claim 25 is analyzed and 
rejected as previously discussed with respected to claim 1 above 

Regarding claim 26, Gvili discloses the computing device is a game console (game 
input device, page 572, section 5). 

Regarding claim 27 and 28, these claims recite same limitations as claims 22 and 23, 
respectively. Thus they are analyzed and rejected as previously discussed with respect to 
claims 22 and 23 above. 

Regarding claim 29, Gvili discloses the video capture device is a webcam (ZCam is 
used to generate depth keying during live broadcasts, page 568, line 5 and page 572, 
section 5). 

Regarding claims 30 and 32, these claims recite same limitations as claim 11. Thus 
they are analyzed and rejected as previously discussed with respect to claim 1 1 above. 

Regarding claim 31, Gvili discloses the scene image data includes an image of a 
person, the image of the person being incorporated into a video game for interaction 
therein (camera using depth-sensing technology are used in man-machine interactions) ( 
second paragraph of Summary, page 572). 
10. Claim 2 is rejected under 35 U.S.C. § 103(a) as being unpatentable over Gvili in view of 
Parker, and further in view of Tuomi, US 7,061,507. 

Regarding claim 2, note the discussion of claim 1 above. Gvili and Parker do not teach 
storing the depth mask in memory of the image capture device. However, Tuomi teaches 
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storing the depth mask in memory of the image capture device (provided a Z-buffer 1902 for 
storing the Z-values relating to the depth of the pixel; see 12:15-36, Tuomi). 

It would have been obvious to one of ordinary skill in the art at the time this invention 
was made to have used a memory as taught by Tuomi as modified by Gvili and Parker so that 
it provide multiple buffers for storing information (12:15-17, Tuomi). 

Conclusion 

11. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: Gajewska (US 5,914,723), Wang et al. (US 5,557,684), Azuma et al. (US 
2003/0012277), Kopec et al. (US 5,706,364), Berman et al. (US 6,134,346), Long (US 
7,161,634), Okada et al. (US 7,283,679), and Chen (US 6,556,704). 

12. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy 
as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS 
from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until 
after the end of the THREE-MONTH shortened statutory period, then the shortened 
statutory period will expire on the date the advisory action is mailed, and any extension fee 
pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of the advisory action. 
In no event, however, will the statutory period for reply expire later than SIX MONTHS 
from the mailing date of this final action. 
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13. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Kent Wang whose telephone number is 571-270-1703. The examiner 
can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ngoc-Yen Vu can be reached on 571-272-7320. The fax phone number for the 
organization where this application or proceeding is assigned is 571-270-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from cither Private PAIR or Public PAIR. Status information 
for unpublished applications is available through Private PAIR only. For more information 
about the PAIR system, see http://portal.uspto.gov/external/portal/pair. Should you have 
questions on access to the Private PAIR system, contact the Electronic Business Center 
(EBC) at 866-217-9197 (toll-free)? If you would like assistance from a USPTO Customer 
Service Representative or access to the automated information system, call 800-786-9199 (IN 
USA OR CANADA) or 571-272-1000. 



/KW/ 

13 Aug 2008 



/Ngoc-Yen T. VU/ 
Supervisory Patent Examiner, Art Unit 2622 



